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1  Executive  Summary 

This  Phase  I  effort  sought  to  develop  a  passive,  electric-optical  and  infrared  (EO-IR)  imaging 
system.  This  system  employs  jointly  optimized  multi-modal  image  acquisition  and  processing 
methods  in  order  to  increase  the  operational  range  (the  distance  between  the  sensor  and  the 
object/scene  of  interest)  by  lOX  over  a  baseline  range  that  corresponds  to  traditional,  single¬ 
mode  image  acquisition  in  the  presence  of  obscurants.  Because  the  gain  in  image  quality 
associated  with  a  single  optimization  technique  may  be  modest,  a  multi-modal  hardware  solution 
combined  with  coordinated  processing  techniques  was  tested  during  this  effort  in  order  to 
achieve  an  exponentially  more  substantial  improvement. 

In  order  to  validate  this  concept,  we  first  developed  a  model  of  the  interaction  between  light  and 
the  scattering  particles  present  in  the  obscurant.  In  the  meantime,  a  laboratory  setup  was  built  to 
validate  experimentally  this  model  and  adjust  our  dehazing  approach. 

The  dehazing  solution  obtained  combines  smart  image  acquisition  and  post  processing  methods. 
It  is  performed  in  three  successive  steps:  spectral  selection  (acquisition),  polarization  modulation 
(acquisition)  and  image  filtering  (post-processing). 

Several  outdoors  measurements  were  conducted  with  a  polarization  imaging  camera  operating  in 
the  visible  spectrum  in  order  to  validate  the  approached  described  above.  Eurther  work  has  also 
been  conducted  in  the  Short  Wave  Infrared  (SWIR)  spectrum.  A  Phase  IB  and  Phase  II  effort 
will  extend  this  effort. 
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2  Project  main  Objective 

The  main  goal  of  this  Phase  1  effort  was  to  determine  the  feasibility  and  design  of  a  multi-modal 
EO-IR  system  with  jointly  optimized  sensing  and  processing  techniques  that  could  achieve  a 
lOX  improvement  in  operational  range  compared  to  single-mode  operation  in  the  presence  of 
obscurant. 

3  Project  Approach 

The  approach  selected  to  reach  this  objective  is  based  on  the  excellent  performance  of  the 
existing  passive  full  Stokes  polarization  imaging  camera  for  visible  light  “SAESA”  (developed 
by  Bossa  Nova  Technologies  DoD  Phase  II  SBIR:  W56HZV-05-C-0646),  coupled  with  that  of  a 
SWIR  (Short  Wave  Infrared)  camera.  The  selected  design  combines  a  polarization  imaging 
camera  operating  in  the  visible-NIR  (Near  Infrared)  range  with  a  polarization  imaging  camera 
operating  in  the  SWIR  (Short  Wave  Infrared)  spectral  band. 

To  validate  the  proposed  approach,  the  Phase  1  effort  was  composed  of: 

Taskl:  Assembly  of  a  laboratory  setup  (completed) 

Task2:  Development  of  the  scattering  model  (completed) 

Tasks :  Processing  development  for  the  visible  polarimetric  camera  software  (completed) 
Task4:  Characterization  of  the  visible  light  polarimetric  system  efficiency  (completed) 
Tasks :  SWIR  camera  for  imaging  through  haze  (completed) 

Task6:  Study  of  the  performance  and  characterization  of  the  SWIR  camera  and  the 
polarimetric  visible  light  range  camera  (completed) 

The  results  obtained  during  this  Phase  I  effort  are  presented  in  five  different  sections  which  have 
been  completed  throughout  the  course  of  those  tasks: 

The  physical  model  of  interaction  light/particles  (Task2) 

This  section  describes  the  model  of  interaction  between  light  and  the  scattering  particles. 
The  simulation  results  obtained  are  detailed,  particularly  the  spectral  dependency  and 
effects  on  the  polarization  of  light.  It  also  highlights  how  the  comprehension  of  these 
interactions  lead  to  a  solution  based  on  a  multispectral,  polarization  imaging  system. 

-  A  Multispectral  polarization  imaging  solution  (Task3) 

In  this  section,  the  steps  leading  to  image  dehazing  are  detailed.  Specific  question  are 
addressed  such  as:  what  polarization  modulation  should  be  used  to  obtain  the  highest 
dehazing  effect?  What  is  the  image  processing  principle?  This  solution  is  evaluated  using 
various  testing  environments. 

The  Testing  environment  (Taskl) 

In  this  section,  controlled  and  uncontrolled  environments  used  to  evaluate  the  efficiency 
of  the  solution  proposed  are  introduced.  A  quick  introduction  on  evaluation  techniques 
challenges  is  also  presented. 

The  results  on  controlled  environment  (Task3+4) 
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This  section  details  the  dehazing  results  obtained  with  the  polarimetric  visible  camera 
using  the  laboratory  setup  developed  in  task2.  In  order  to  quantitatively  report  the 
dehazing  effect,  one  evaluation  technique  is  introduced. 

The  results  on  non-controlled  environment  (Task5+6) 

This  section  details  the  dehazing  results  obtained  with  the  polarimetric  visible  camera  and 
the  SWIR  camera  on  the  field.  Another  method  to  quantify  the  dehazing  effect  is 
introduced. 


3.1  Physical  model  of  interaction  light/particles 


Figure  1:  Schematic  representation  of  the  interaction  ofiight  on  dear  day 


During  a  clear  day,  an  imaging  system  can  collect  principally  two  light  components  (figure  1): 


^detect^^>y  >  ^obiect^^>y  ^object  ^^>y  >  ^  (t) 

Idetecti^r  y>  (p)  the  light  impinging  on  the  camera 

I objecti^’Y’  (p)  the  light  from  the  sun  reflected  by  the  object  towards  the  camera  direction 

Eobject(^>  y>  (p)  the  light  emitted/radiated  spontaneously  by  the  target  object 

(x,  y)  the  pixel  coordinates  of  the  scene, 

1  the  wavelength  of  light  observed  using  the  camera, 

(9,  cp)  the  angles  in  spherical  coordinates  of  light  illumination 


During  a  foggy  day,  small  liquid  droplets  present  in  the  air  scatter  the  light  in  the  atmosphere.  As 
a  result,  visible  light  coming  from  the  object  observed  is  attenuated  and  the  image  observed  is 
blurred.  The  ambient  light  path  is  also  modified  by  those  aerosolized  particles:  atmospheric  light 
scattered  by  the  obscurant  medium  combines  with  reflected  light  from  the  target  objet  causing  a 
decrease  in  image  contrast  as  well  as  color  degradation  (figure  2). 


Figure  2:  Schematic  representation  of  the  interaction  ofiight  with  aerosoiized  particie  during  imaging 
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The  radiative  transfer  equation  for  the  intensity  of  light  measured  on  the  detector  can  be 
expressed  as: 

^detect  y  j  ^  Zj^')  ^scat  [object]  y  "t  ^scat  [particles]  ^^>y  >  ^  (  2  ) 

^object  {x,y,X, 6, ip,t,z,r)  ^particles  i?^>y >  ,  (p ,t,  Z,t) 

With 

hcat[object](.^’y’^’0’<P’i’^’f)  the  light  comiug  from  the  object  illuminated  by  a  natural  light 
source  /  object  which  travels  through  the  medium  and  is  scattered  by  haze  particles  during 
its  path  towards  the  camera 

heat  [particles]  (^>  y>  <P>  the  backscattered  light  also  called  airlight  the  light  coming 

from  the  illumination  source,  scattered  by  the  medium  and  going  directly  to  the  camera 
Eobjecti^’  y>  (P>  the  light  emitted/radiated  by  the  target  object 

Eparticie(^>y>^>^>(P>i>^>'‘^)  the  light  emitted/radiated  by  the  aerosol  particles  located  between 
the  object  and  the  sensor 

(t,  z,  r)  are  the  scattering  medium  characteristics.  They  are  respectively  the  medium 
transmission,  the  haze  depth  and  the  particle  size. 


In  most  cases,  the  light  spontaneously  emitted  by  the  observed  object  and  by  the  aerosol  particles 
is  the  result  of  thermal  radiations.  The  wavelength  of  the  light  emitted  depends  on  the 
temperature  and  can  be  calculated  using  Planck's  law  of  black-body  radiation  Using  the 
Wien’s  approximation,  the  maximum  wavelength  peak  of  this  radiation  X  is  given  by 


_'2 

With  b  the  Wien's  displacement  constant  equal  to  2.898.10  m.K  and  T  the  temperature  in 
Kelvin.  Using  this  relation  it  is  possible  to  estimate  the  typical  light  contribution  from  haze  or 
typical  passive  object. 

For  this  type  of  objects,  the  typical  temperature  is  around  293-323K  and  thus,  the  peak  radiation 
is  mostly  in  the  far  infrared 9pm. 

With  these  considerations.  Equation  (3)  can  be  simplified  for  the  visible  [400-700nm],  near 
infrared  (NIR)  [700-1000nm]  or  short  wave  infrared  (SWIR)  [l-3um]  wavelengths  to  the 
following  expression: 

hetect  (yt  ,y ,  X,6 ,  (p  ftfZfVj  Xcat  [ob/ect]  t,  Z,  r)  -t  I  scat  [particles]  (pt ,  y ,  X,  6 ,  (p ,  t ,  Z ,  vj  (  4  ) 


The  goal  is  to  estimate  I  object  with  the  best  accuracy.  In  order  to  recover  lobject  from  the 
intensity  detected  Idetect  ?  it  is  necessary  to  minimize  the  airlight  component  Igcat  [particles] 
relatively  to  the  signal  of  interest  lobject  • 
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The  undesired  parameter  I^cat  [particles]  does  not  have  the  same  physical  origin  as  lobject-  Thus, 
scattered  photons  can  have  different  dependency  regarding  some  physical  parameters  (such  as 
polarization)  compared  to  ballistic  photons  directly  reflected  by  the  object. 

Therefore,  the  key  to  achieve  a  lOX  improvement  in  operational  range  compared  to  single¬ 
mode  operation  in  the  presence  of  obscurant  is  to  understand  those  dependencies.  This  has 
been  done  by  modeling  the  interaction  between  light  and  haze  particle. 

The  next  section  introduces  the  model  used.  It  details  also  the  principal  results  that  have  been 
obtained  through  it:  the  undesired  parameter’s  amplitude  depends  on  the  light’s  wavelength 
and  features  a  specific  polarization  signature.  Those  results  have  been  used  to  design  the  best 
imaging  acquisition  system  possible. 

3.1.1  Light  scattering  models 

As  discussed  previously,  the  presence  of  aerosolized  particle  introduce  an  undesired  parameter 
Iscat  [particles] -This  component  is  due  to  the  interaction  between  light  and  haze  particle:  when 
light  interacts  with  matter,  the  electromagnetic  field  is  modified  and  the  photons  can  be 
reradiated  (scattered)  by  the  particle  towards  the  camera  direction. 

This  interaction  can  be  classified  in  two  categories:  elastic  and  inelastic  {the  elastic  scattering,  in 
contrast  to  the  inelastic  scattering,  is  a  process  in  which  the  kinetic  energy  of  the  light  is 
conserved}.  The  elastic  scattering,  which  is  detailed  in  this  study,  is  the  predominant  effect  in 
scattering  (approximately  only  1  out  of  lO^photons  scattered  is  not  scattered  elastically) 

The  elastic  scattering  effects  can  be  described  using  two  different  interaction  models:  the 
Rayleigh  and  the  Mie  theories.  The  description  of  the  scattering  process  with  one  or  the  other 
theory  is  linked  to  the  circumference  of  the  scattering  particle  tiD  (D  the  diameter  of  the  particle) 
and  the  wavelength  of  the  incident  light  X. 

•  ttD  «  A  :  Rayleigh  scattering 

In  the  Rayleigh  scattering  model,  the  scattering  particle  can  be  seen  as  an  oscillating  dipole 
induced  by  the  light  electric  field.  Thus,  the  dipole  radiation  pattern  can  be  calculated  (see 
detailed  in  Appendix  B-  Rayleigh  scattering). 

•  ttD  ~  A  :  Mie  scattering 

The  Mie  theory  is  a  very  general  and  complex  scattering  model,  which  describes  the  interaction 
of  an  electromagnetic  plane  wave  by  a  single  homogeneous  sphere  using  the  Maxwell  equations 
(see  detailed  in  Appendix  B-  Mie  scattering).  It  is  valid  for  any  particle  size.  However,  it  is 
harder  to  apprehend  directly  the  concept,  which  explains  why  the  Rayleigh  theory  is  generally 
preferred  for  small  sized  particles  compared  to  wavelength. 

Since  Rayleigh  scattering  approach  is  an  approximation  of  the  Mie  scattering  model,  it  is 
therefore  necessary  in  the  general  case  to  develop  a  complete  Mie  scattering  model  software  to 
predict  the  haze  effect  on  measurements  (see  Appendix  B-  Mie  scattering). 
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In  fog,  the  ambient  medium  is  air  with  a  permeability  of  1.25663753x10”^  H/m  and  the  haze 
particle  is  principally  composed  of  water  with  a  permeability  of  1.256627x10”^  H/m  Thus,  we 
can  in  first  approximation  consider  that  the  ratio  of  magnetic  permeability  is  equal  to  1 . 

Under  that  approximation,  only  three  parameters  are  necessary  to  determine  completely  the 
scattered  light  field  Iscat  [particles]: 

•  The  size  of  the  haze  particle:  r  =  D/2 

•  The  complex  index  of  refraction  of  the  haze  particle:  m 

•  The  electric  field  of  the  incident  light. 

Note:  the  dependency  of  the  scattering  with  the  wavelength  is  not  clearly  visible  with  the  Mie 
theory  but  is  included  through  the  dependency  of  the  refraction  index  with  the  wavelength 
(Cauchy  formula,  Briot  dispersion  or  equivalent). 


The  size  of  the  haze  particle:  r  =  D/2 

Fog  particles  have  typically  a  radius  of  r=0.01um  to  r=lum.  Thus,  in  the  simulation  made  the 
dependency  of  the  scattered  field  is  simulated  with  three  different  particle  sizes:  0.01;  0.1  and 
lum.  We  will  also  observe  the  result  electric  field  if  we  have  fog  composed  with  different 
particle  size. 


The  complex  index  of  refraction  of  the  haze  particle:  m(A) 

As  noted  above,  the  complex  index  of  refraction  depends  on  the  wavelength.  In  order  to  have 
good  simulation  results,  it  is  necessary  to  have  the  most  accurate  measure  for  this  parameter.  To 
match  better  the  reality,  we  will  use  the  previous  database  defined  by  the  International 
Association  for  the  Properties  of  Water  and  Steam  which  has  been  tested  for  a  wavelength  range 
up  to  1900nm  In  their  formulation,  the  real  part,  n,  of  the  refractive  index  of  water,  m=n+ik, 
is  expressed  as: 


rd  -  1 
+  2 


P_ 

P* 


p  T  XT  /X\  ^ 

“o  +  “l  ^  +  “2  ^  +  “3  +  “4  + 


(5) 


With  the  following  constants  values: 

ao=0.244257733;  ai=9.74634476.10-^;  a2=-3.73234996.10-^;  a3=2.68678472.10'^ 
a4=l. 58920570. 10'^;  a5=2.45934259.10'^;  a6=0.900704920;  a?  =  -1.66626219.10'^;  /.uv  = 
0.2292020;  /.ir=5.432  937 


and  the  dimensionless  variables: 

•  temperature:  T/T*  (where  T*=273.15  K) 

•  density:  p/p*  (where  p*=1000  kg.m'  ) 

•  wavelength:  X/  X*  (where  X*=  0.589  pm) 
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The  problem  of  this  formulation  is  that  it  does  not  provide  the  imaginary  part  of  the  refractive 
index  of  water  (usually  estimated  as  0  in  visible  range)  which  could  be  crucial  for  scattering 
understanding. 

For  this  reason,  in  addition  to  this  database  used  to  determine  the  real  coefficient  of  the 
refractive  index,  another  one  was  employed  for  the  imaginary  coefficient  of  water 
determination. 

The  electric  field  polarization. 

The  light  coming  directly  from  the  sun  is  un-polarized.  In  other  world,  it  is  combination  of 
arbitrary  polarized  light  with  arbitrary  phase  (stochastic  distribution  of  polarization).  To 
simulate  with  the  program  the  effect  of  this  light,  a  Monte-Carlo  simulation  was  used:  the 
program  will  generate  hundreds  of  electric  field  with  completely  arbitrary  incident  polarized 
light  and  the  scattering  wave  resulting  of  all  those  contributions  is  observed  in  far-field. 

3.1.2  Simulation  results 

As  discussed  previously  (see  Eq.  4),  the  goal  is  to  minimize  the  airtight  component 
Iscat  [particles]  relatively  to  the  signal  of  interest  lobject-  Using  the  model  implemented  and  the 
assumption  detailed  in  Appendix  B-  Mie  scattering,  the  following  important  simulation  results 
on  Teat  [particles]  Can  bc  obscrvcd  (Figure  3  to  5): 

3.1. 2.1  Wavelength  dependency  of  l^cat  \particies] 

The  light  scattered  intensity  (Igcat  [particles])  depends  on  wavelength  (Figure  3) 


wavelength  A  [pm] 


Figure  3:  Evolution  of  the  reduced  scattering  coefficient  with  the  wavelength  from  visible  to  SWIR 
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The  measure  of  the  reduced  scattering  coefficient  for  the  three  particle  size  relative  to  the 
wavelength  is  plotted.  This  coefficient  corresponds  to  the  number  of  scatter  event  N  observed 
during  the  propagation  through  a  homogeneous  scattering  medium  with  a  thickness  d.  In  other 
word,  if  the  reduced  scattering  coefficient  is  high,  there  is  more  haze  effect. 

For  the  smallest  particles  this  factor  decreases  drastically  with  the  wavelength  [Note  that  we  can 
retrieve  the  factor  X  predicted  in  Rayleigh  regime  range].  For  example  for  a  particle  with 
r=0.1pm,  using  this  simulation  we  can  deduce  at  a  wavelength  of  1500nm  (SWIR)  compared  to  a 
wavelength  of  500nm  (visible)  a  reduction  of  the  scattering  events  by  100.  Thus,  haze  effect  will 
be  GENERALLY  less  important  for  higher  wavelength.  Particularly,  for  the  visible 
spectrum,  the  effect  will  be  lower  in  the  red  than  in  the  blue  and  less  important  in  SWIR 
than  in  the  visible  spectrum. 

The  last  data  plotted  in  this  simulation  (r=0.01pm)  is  really  important.  We  can  see  that  the 
behavior  on  this  case  is  not  the  one  expected  in  the  Rayleigh  regime.  In  this  case,  for  specific 
wavelengths  around  400nm  (violet-blue  light),  scattering  effects  are  expected  to  be  less 
important  than  higher  wavelength  such  as  red  (around  600nm)  or  even  SWIR  (around  lum). 

Thus,  we  can  conclude  using  this  graphic  that  it  is  important  to  have  a  multispectral  acquisition 
system  (which  is  part  of  the  solution  proposed). 


3. 1.2.2  Polarization  signatures  of  Iscat[particies]iPid^'^^ 

If  scattered  light  has  specific  polarization  signature  (due  to  the  interaction  between  light  and  the 
aerosolized  particle),  there  is  no  reason  that  the  un-polarized  sunlight  directly  reflected  by  the 
object  will  be  the  same.  Thus,  it  is  possible  to  minimize  the  airtight  component  l^cat  [particles] 
relatively  to  the  signal  of  interest  lobject  by  using  this  polarization  information  (see  the 
Multispectral  polarization  imaging  solution  section  for  detailed  on  the  dehazing  method). 
Hereafter,  we  demonstrate  experimentally  this  specific  polarization  signature  of  the  light 
scattered. 

a.  The  polarization  signature  of  the  scattered  light ’s  can  be  partially  linear  (Figure  4) 
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0  [rad] 

Figure  4:  Evolution  of  the  Degree  Of  Linear  Polarization  with  the  angle  of  scattering  for  non-poiarized  light  at 

SOOnm 

The  theoretical  value  for  the  Degree  Of  Linear  Polarization  (DOLP)  using  the  Rayleigh  model 
(black  line)  is  compared  to  the  one  obtained  with  the  Mie  scattering  model  program  described 
above.  Three  different  radii  sizes  of  particle  r=0.01;  0.1;  lum  and  non-polarized  light  at 
7,=500nm  are  considered.  The  Rayleigh  scattering  model  is  a  good  approximation  for  small 
particle  size  compared  to  wavelength.  Thus,  we  observe  as  the  theory  previse  that  the  DOLP 
simulation  result  using  Mie  theory  is  similar  to  the  result  obtained  with  Rayleigh  model  for  the 
smallest  particle  of  O.Olum  and  differs  from  it  for  the  biggest  particle  of  lum. 

From  this  graph,  we  can  conclude  that  the  scattered  light  has  a  complex  polarization  signature 
that  depends  on  different  parameters  such  as  the  angle  of  scattering  (angle  between  the  initial  and 
final  directions  of  light  propagating  through  the  scattering  particle)  and  the  particle  size.  Thus, 
the  polarization  information  can  be  used  to  differentiate  the  scattered  photon  from  the  ballistic 
photon  directly  reflected  by  the  object. 

We  can  conclude  that  it  is  important  to  have  a  polarimetric  acquisition  system  (which  is  part  of 
the  solution  proposed). 

b.  The  polarization  signature  of  the  scattered  light ’s  can  be  partially  linear,  circular 
or  elliptical  (Figure  5) 
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Figure  5:  Evolution  of  the  polarization  with  the  angle  of  scattering  (r=0.1pm) 

The  evolution  of  the  normalized  Stokes  (si,  si,  S3)  vector  depending  on  the  angle  of  scattering  is 
plotted  for  a  particle  with  r=  0.1pm  and  an  incident  light  wavelength  A  =1500nm. 

In  figure  5,  we  can  observe  that  the  parameter  Si  fluctuates  depending  on  the  angle  of  scattering. 
Because  the  DOLP  is  expressed  out  of  si  and  S2  (see  appendix  A),  it  also  fluctuates  (as  it  has 
been  shown  in  figure  4). 

It  can  also  be  noted  that  the  presence  of  circularly  polarized  light  (S3),  which  was  not  predicted 
by  the  Rayleigh  model.  The  circular  polarization  is  useful  not  only  because  it  provides  additional 
data  about  the  fog  presence,  but  also  because  it  is  maximized  when  the  linear  components  (si  and 
S2)  are  at  their  minimum.  In  other  words,  information  on  the  scattered  light  can  be  obtained  for 
any  scattering  angle  if  polarization  data  is  used  properly. 

We  can  conclude  from  this  graph  that  it  is  important  to  have  a  full  Stokes  polarimetric 
acquisition  system. 

All  the  simulation  runs  and  the  results  obtained  (figure  3  to  5)  lead  to  an  acquisition  system 
based  on  a  multispectral  polarization  imaging  system.  The  next  section  will  detail  this 
solution. 
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3.2  Multispectral  polarization  imaging  solution 

The  multispectral  polarization  imaging  solution  proposed  is  a  complete  dehazing  method  based 
on  three  consecutive  steps  (see  figure  6)  that  will  take  advantages  of  the  information  discussed  in 
the  previous  section: 


•  Two  acquisition  steps  (a  spectral  selection  and  a  polarization  modulation)  in  order  to  acquire 
the  image  with  as  few  haze  effects  as  possible. 

•  One  post  processing  step  on  the  image  in  order  to  complete  image  dehazing 


Figure  6:  Basic  architecture  of  the  muitispectrai  poiarization  imaging  soiution 


3.2.1  First  step:  Spectral  selection  [acquisition] 

As  it  has  been  pointed  out  during  the  simulation,  the  scattering  effect  depends  highly  on  the 
wavelength  considered  (chromaticity  dependency). 

For  this  reason,  in  order  to  minimize  haze  effect,  a  spectral  selection  is  used.  This  technique 
consists  in  observing  the  same  scene  with  various  spectral  filters  and  selects  the  filter  for  the 
image  acquisition  where  the  haze  effect  is  less  present  (Igcat  [particles]  minimized)  and  the  signal 
of  interest  is  the  strongest  (lobject  maximized). 

In  practice,  this  was  done  in  the  experiments  during  this  Phase  I  effort  by  tuning  the  light  source 
or  by  acquiring  the  same  scene  while  changing  manually  the  spectral  filter  in  front  of  the  camera 
or  using  the  SWIR  instead  of  the  visible  spectrum  camera  (on  the  field  experiments). 

In  order  to  determine  which  were  the  images  with  less  haze  effects,  various  methods  were 
explored  which  will  be  detailed  on  the  results  sections. 


3.2.2  Second  step;  Polarization  modulation  [acquisition] 

The  light  scattered  by  haze  particles  can  present  strong  and  very  specific  polarization  signature. 

In  most  cases  the  object  observed  does  not  feature  the  same  polarization  signature.  Therefore, 
polarization  imaging  capabilities  can  be  a  very  efficient  tool  to  isolate  and  reject  the  light 
contribution  due  uniquely  to  the  haze  particles.  Isolating  that  part  is  the  first  step  to  then 
attenuate  it,  and  in  some  cases  completely  remove  the  blurring  effect  on  the  image. 

But  in  order  to  do  so,  it  is  necessary  to  know  precisely  the  polarization  properties  of  the  light 
reflected  by  the  object  directly  toward  the  camera.  The  general  assumption  made  is  that  light 
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directly  reflected  by  the  object  is  not  polarized.  This  approximation  is  valid  for  objects  with 
diffusive  surface.  It  is  mainly  not  valid  for  objects  featuring  specular,  highly  reflective  surfaces. 
However,  the  more  scattering  particles  are  present,  the  lower  the  portion  of  light  coming  directly 
from  the  object  becomes  compare  to  airlight  scattered  by  the  obscurant. 

It  has  been  shown  that  for  an  object  far  enough  (Pz  >  0.2  with  p  coefficient  of  extinction 
due  to  scattering  and  absorption  and  z  the  distance  between  observer  and  object),  this  general 
assumption  becomes  valid,  even  for  specular  surfaces  (see  figure  26  for  experimental 
validation).  In  other  words,  in  presence  of  obscurant,  the  light  reflected  by  an  object  far  enough 
can  be  considered  un-polarized. 

The  dehazing  method  becomes  easy  to  understand  by  looking  back  at  Equation  (4).  This 
equation  can  be  rewritten: 

I  _  /  .  4_  /  4_  /  unpol  ( 6 ) 

‘detect  ‘scat  [object]  '  ‘  scat  [particles]  '  ^  scat  [particles] 


The  light  detected  is  the  superimposition  of: 

•  Iscat  [object]  >  unpolarized.  This  is  what  we  need  to  isolate. 

•  Iscat  [particles]  >  partially  polarized.  This  is  the  contribution  due  to  the  obscurant  particles, 
which  need  to  be  minimized. 

It  is  decomposed  as  /  [particles]  and  /  "^Z^lparticies]  wich  are  respectively  the 
component  of  the  scattered  light  completely  and  not  polarized. 

In  order  to  minimize  the  second  term  of  Equation  4,  three  different  techniques  based  on 
polarization  imaging  were  explored  during  this  Phase  I  effort  (see  Appendix  B-polarization 
modulation  for  details): 

3.2.2. 1  Using  a  "linear  polarizer" 

The  idea  is  to  add  a  linear  polarizer  filter  on  the  light’s  pathway,  whose  axis  orientation  is 
adjusted  in  order  to  cut  part  of  the  polarized  light  coming  from  the  haze  particles  scattering. 

Although  attractive  because  of  its  simplicity,  this  technique  only  produces  limited  results.  This  is 
due  to  two  reasons: 

•  Light  scattered  is  only  partially  polarized 

•  The  polarized  part  of  the  scattered  light  is  not  purely  linearly  polarized,  except  in  rare 
configuration  (see  figure  4).  Most  times  it  is  partially,  elliptically  polarized. 

The  consequence  is  that  a  linear  polarizer  cannot  remove  completely  the  contribution  of  scattered 
light. 
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3.2.2.2  Using  the  measured  Degree  Of  Polarization 

Since  the  assumption  made  is  that  the  light  coming  from  the  target  is  un-polarized,  one  method  consists 
of  removing  all  the  polarized  part  of  the  light.  By  using  a  polarization  camera,  we  can  extract  the  DOLP 
and  thus  separate  the  image  into  its  totally  polarized  part  and  un-polarized  part.  The  totally  polarized 
part  can  be  excluded  to  recover  only  the  light  coming  from  the  target.  Again,  there  is  a  limitation  with 

this  approach.  This  technique  removes  only  the  polarized  component  I  ^  [particles] 
polarized  component  I  “nparticiesjof  *e  light  scattered  I^cat  [particles]  (see  Eq.6). 

3.2.23  Using  the  extended  Degree  Of  Polarization  extracting  technique 

This  technique  is  an  extension  of  the  previous  method.  It  is  based  on  three  steps: 

1-  Measuring  the  polarized  part  of  the  scattered  light:  I  [particles] 

This  can  be  done  by  using  the  technique  described  in  the  section  above. 

2-  Measuring  the  ratio  I  ^Z^artides]/ ^  IcL  [partidesfn  the  scene 

The  additional  assumption  made  is  that  the  scattering  angles  (the  angle  between  the  light  source, 
the  haze  particle  and  the  line  of  sight)  and  the  nature  of  the  particles  are  constant  within  the  scene 
observed.  As  a  consequence,  the  ratio  between  the  un-polarized  and  polarized  parts  of  the  light 

scattered  [I  scT[particies]/I  St  [particles]]  the  obscurant  remains  constant  as  well.  In  other 
words,  the  only  variations  observed  within  the  scene  are  due  to  the  difference  of  fog  density 
between  the  scene  and  the  observer. 

This  ratio  can  be  measured  on  a  known  point  where  only  airlight  is  present  (clear  sky  for 
example). 

3-  Remove  completely  hmt  [pm-firi^.^]  on  each  pixel  of  the  scene 

^  St  [particles]  measured  via  the  measurement  of  the  Degree  Of  Polarization  (DOP)  for 

each  pixel  of  the  scene. 

The  ratio  I  scT[particies]/I  St  [particles]  ^as  also  been  estimated  and  is  assumed  to  remain 
uniform  across  the  image. 

Therefore,  I  scaSparticies]  estimated  for  each  pixel  of  the  scene. 

We  now  have  estimated  the  two  contributions  of  the  total  light  scattered  by  the  haze  particles,  its 
polarized  and  its  un-polarized  parts:  I^cat  [particles]  =  I  St  [particles]  +  ^  SSpartlcles] 
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3.2.3  Third  step:  Post  processing  techniques 

Once  images  have  been  acquired  at  a  specific  wavelength(s)  and  polarization  imaging  techniques 
have  been  applied,  the  last  step  is  to  apply  well-known  image  processing  tools. 

The  model  widely  used  for  post-processing  techniques  to  describe  hazed  images  is 

ldetect(x,y)  ^  J(x,y)t(x,y')  +  V(x,y')  (7) 

With  Idetect  the  intensity  detected  by  the  camera,  J  the  scene  radiance,  t  the  medium  transmission, 
and  V{x,y)  =  A(1  —  the  atmospheric  veil  where  A  is  the  global  atmospheric  light,  and 

(x,  y)  is  the  pixels  coordinates  of  the  scene. 

The  problem  with  this  model  is  that  it  is  a  dependent  system  of  equation:  there  are  three 
unknown  parameters  for  only  one  equation  (infinity  of  possible  solutions).  Thus,  in  order  to 
solve  this  problem,  some  additional  physical  constraints  need  to  be  applied. 

The  main  assumption  used  is  to  consider  that  the  atmospheric  veil  (and  consequently  the 
transmission  map)  has  no  specific  edges  or  textures. 

So,  in  order  to  estimate  the  haze  layer  or  atmospheric  veil  from  the  hazy  image,  a  low  pass  filter 
is  applied  to  the  image.  The  equation  system  is  composed  of  two  independents  equations  with 
three  independent  unknown  parameters: 

Uetectix.y)  ^Jix,y)t(x,y)  +  V(x,y) 

LowPassQaetectix.y))  =  V(x,y) 

Then,  in  order  to  fully  solve  the  problem,  the  general  technique  involves  determining  the  airlight 
parameter  A  by  measuring  it  in  area  where  it  is  perfectly  known  such  as  sky  where 

^detecti^^skyiysky)  —  A  (9) 

This  way,  the  transmission  map  can  be  recovered  and  thus,  the  intensity  of  the  dehazed  image. 
However  the  transmittance  map  recovered  suffers  from  the  strong  assumptions  made  which  are 
not  perfectly  verified  on  whole  image  or  for  all  geometrical  configurations  of  objects  in  the 
scene.  Additional  filters  are  generally  used  to  smooth  the  obtained  transmittance. 

The  basic  principle  is  summarized  in  figure  7  (for  more  details  please  refer  to  Appendix  B-Post- 
processing  techniques). 
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Figure  7:  Basic  architecture  of  post-processing  technigues  to  recover  an  image  dehazed  from  a  raw  image 


This  first  study  was  limited  on  the  case  of  dark  channel  prior  (DCP)  technique  (for  more 
details  please  refer  to  appendix-Post-processing  techniques)  which  presents  the  combination  of 
relatively  good  efficiency  and  computational  speed 


3.3  The  T esting  environment 

In  order  to  evaluate  and  validate  the  dehazing  system,  two  completely  different  environments 
have  been  tested:  a  fully  controlled  lab-setup  (in  order  to  validate  the  methodology  and  the 
concept)  and  a  non-controlled  environment  (in  order  to  validate  the  performances  in  real 
condition). 

3.3.1  Fully  controlled  lab-setup 

For  the  laboratory  setup,  two  main  constraints  have  been  followed:  1-  The  laboratory  setup 
should  closely  mimic  the  behavior  of  fog  in  natural  condition,  2-  it  is  necessary  to  have  a 
complete  control  of  the  experimental  setup. 

The  fully  controlled  experimental  testing  setup  is  composed  of  a  transparent  tank,  a  scattering 
medium,  an  illumination,  the  Salsa  Full  Stokes  polarimetric  camera  and  a  target  (see  Figure  8). 
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Figure  8:  Experimental  testing  setup 


The  transparent  tank 

The  principle  of  having  a  transparent  tank  is  to  be  able  to  control  the  quantity  of  scattering 
particles.  To  do  so  at  low  price,  the  solution  adopted  was  to  use  an  object  specially  designed  for 
it:  an  aquarium.  There  are  mainly  two  different  types  of  aquarium  enclosure:  acrylic  and  glass. 
Acrylic  element  (such  as  CD  case  have  strong  polarization  signature.  Thus,  it  is  not  a  good 
enclosure  to  test  the  polarization  modulation  dehazing  method  developed.  For  this  reason,  the 
transparent  tank  used  is  a  glass  aquarium. 

The  scattering  medium 

In  order  to  mimic  fog  behavior,  many  different  techniques  can  be  employed:  fog  machine, 
milk+water,  fog  effect  glass  filter. . . 

A  fog  machine  presents  the  advantage  to  look  like  the  fog  that  could  appear  in  outdoor 
environment.  But  using  directly  a  fog  machine  can  be  challenging  at  this  validation  step  since  it 
is  not  proven  that  a  manmade  fog  will  have  the  same  behavior  as  fog  in  natural  condition  in 
every  aspect.  Many  factors  are  also  not  under  control  such  as  the  size  of  the  particles,  the  exact 
composition,  the  distribution  in  space,  the  concentration... 

On  the  other  hand,  a  fog  effect  glass  filter  is  exactly  under  control  but  has  no  degree  of  freedom 
(for  example  no  modification  of  “fog”  concentration  possible). 

In  haze,  the  typical  size  of  the  particles  is  lO-lOOOnm.  The  whole  milk  usually  contains 
approximatively  2.7%  of  casein  (~  0.04-0.3  pm)  and  3.4%  of  fat  globule  (1-10  pm)  Due  to 
their  size  and  index  of  refraction,  milk  -i-  water  solution  is  usually  used  to  mimic  the  effects  of 
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fog  scattering.  This  technique  was  used  here  to  validate  our  system  since  it  is  completely 
controllable  and  it  provides  similar  results  as  the  one  encountered  in  fog. 

To  simplify  the  problem  and  consider  only  one  kind  of  particle  in  our  experiment,  low  fat  milk 
(where  only  casein  is  present)  will  be  used  in  the  result  presented.  Three  different  volume 
percentage  of  milk  were  used  0.013,  0.009  and  0.007%. 

Illumination 

To  mimic  the  sunlight,  a  non-polarized  white  light  illumination  was  used.  The  illumination  is 
composed  of  a  grid  of  diodes  (enclosed  in  a  waterproof  case)  and  a  diffuser  (see  figure  9). 


Diodes  in  waterproof  case  Diffuser 

Figure  9:  Illumination 


The  illumination  can  provide  white  light  or  bandpass  red,  green  or  blue  light.  The  waterproof 
case  will  allow  in  the  experiment  to  submerge  the  light  during  the  water  experiments  [This  way, 
there  is  no  partial  polarization  induce  by  light  transmitted  after  an  air-water  interface  that  would 
create  artefact  during  polarization  modulation  method]  .The  light  Diffuser  is  a  diffuser  film 
designed  to  break  up  and  distribute  light  evenly. 

The  illumination  is  placed  on  a  rotating  stage  (figure  10)  so  the  angle  defined  by  the 
illumination,  the  object  and  the  camera  can  be  adjusted  without  modifying  the  target-camera 
distance. 
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Figure  10:  Rotation  stage  for  the  iiiumination.  Some  of  the  configurations  avaiiabiefor  the  iiiumination  are 

presented  in  red  shape. 

Full  Stokes  polarization  imaging  camera  (visible  spectrum) 

The  Salsa  Full  Stokes  camera  is  our  polarization  camera  (figure  11). 


Figure  11:  Salsa  Full  Stokes  visible  camera 

The  objective  lens  used  in  the  experiments  presented  has  a  focal  length  of  25mm.  Bandpass  red, 
green  and  blue  filters  were  used  during  our  laboratory  experiments  and  laser  line  filters  at  400, 
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450,  500,  600,  650nm  with  FWHM=10nm  where  used  for  outdoors  experiments.  For  each  of 
those  filters,  the  camera  has  been  calibrated  by  running  a  process  during  which  several  hundred 
known  polarization  states  are  generated  and  measured  by  the  SALSA  camera.  It  means  that  for 
each  wavelength  used,  we  have  determined  the  calibration  matrix  C  which  links  the  4  raw  frame 
images  acquired  successively  (lo,  h,  I 2,  I 3)  with  the  4  Stokes  parameters  (So,  Si,  S2,  S3)  using  the 
data  reduction  matrix  technique 

s=c^i 

The  target(s) 

Light  reflected  by  objects  can  be  polarized  or  not  depending  on  its  surface  However,  as  we 
have  discussed  in  Multispectral  polarization  imaging  solution-polarization  modulation  section, 
when  the  object  is  far  enough,  the  light  emanating  from  scene  objects  has  insignificant 
polarization  contribution  to  the  measured  polarization. 

To  test  and  verify  this  hypothesis,  two  different  objects  were  used  during  the  laboratory 
experiments  (see  figure  12): 

A  square  spectralon  (from  labsphere):  diffusive  surface. 

A  toy  car:  specular  surface. 


Diffuse  target 


Specular  target 

Figure  12:  Targets 


The  configuration 

Two  different  configurations  have  been  considered  (figure  13). 
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Figure  13:  The  two  different  configurations  used  during  the  experiments.  In  configuration  1,  the  angle  a  can  be 
adjusted  from  40  to  90deg.  In  configuration  2,  the  angle  is  fixed. 

In  the  first  configuration,  the  angle  a  defined  by  the  illumination,  the  target  and  the  camera  can 
be  adjusted.  The  main  component  of  airtight  is  induced  by  forward  scattering  in  small  particles. 

In  the  second  configuration,  the  angle  defined  by  the  illumination,  the  target  and  the  camera  is 
fixed  to  45deg.  The  component  of  airtight  is  induced  by  forward  and  backward  scattering  in 
small  particles. 

This  two  different  configurations  and  the  different  concentration  of  milk  allows  mimicking 
natural  behavior  of  fog  in  various  conditions. 


3.3.2  Non-controlled  environment 

For  this  setup,  we  considered  outdoor  environment.  Two  main  guidelines  have  been  followed  to 
determine  the  scene  to  observe:  1-  The  haze  observed  needed  to  be  similar  as  the  one  that  can  be 
observed  on  shipboard  environment  (therefore,  we  select  a  coastal  scene),  2-  The  scene  should  be 
highly  textured  in  order  to  quantify  dehazing  improvement  (thus,  we  select  a  scene  where  lots  of 
buildings  are  present). 

The  scene  selected  was  the  shore  of  Santa  Monica,  CA.  It  was  observed  from  Manhattan  Beach, 
CA  (figure  14).  The  scene  was  observed  using  the  Salsa  Full  Stokes  polarization  camera 
described  above  and  a  SWIR  non-polarimetric  camera. 
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Figure  14:  Image  of  the  scene  observed  for  on  field  experiments  (Santa  Monica)  and  map  of  the  distance  between 
the  observation  point  (Manhattan  Beach)  and  the  scene  (Santa  Monica) 


a.  SWIR  camera 

The  SWIR  camera  sensor  used  was  an  InGaAs  sensor  which  has  the  best  efficiency  for  this 
bandpass  [l-3um]  (see  figure  15) 


IR  Detectors 


UV-Si  Si  Ge  InGaAs-n  InGaAs-ext  InAs  -30°C 


tnSB..190°C  MCT-6_-30°C  MCT16_.190°C^  -Pyroel  —  -P  per  Photon 
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The  choice  of  the  sensor  itself  has  been  made  to  maximize  the  signal  to  noise  ratio  and  have  a 
high  frame  rate  with  high  resolution  SWIR  camera.  The  main  characteristics  of  the  SWIR 
camera  used  are  presented  in  Table  1. 


Sensor 

SWIR  camera 

Resolution  (pix) 

636  X  508 

Wavelength  range 

900  to  1700nm 

Cell  size  (urn) 

25x25 

Temporal  dark  noise 

400e- 

Dynamic  range 

73dB 

Quantum  efficiency  @  1300 

73% 

Max  Frame  rate 

100  fps 

Table  1:  Main  characteristics  of  the  SWIR  camera 


We  can  note  that  the  camera  sensor  used  has  greater  than  99.5%  operating  pixels.  This  is  an 
excellent  ratio  for  InGaAs  sensors. 

b.  Evaluation  techniques 

Two  types  of  techniques  can  be  implemented  in  order  to  evaluate  dehazing  performances: 
qualitative  and  quantitative  methods  (see  Appendix  A-Performance  metrics  for  more  details). 

Most  of  qualitative  methods  are  made  by  using  human  perception  The  advantage  of  this 
method  is  that  it  is  not  this  way  necessary  to  have  a  strong  knowledge  of  the  dehazed  scene  to 
determine  the  dehazing  method  efficiency.  However,  this  technique  has  many  limitations: 

-  Quality  depends  on  the  observer  perception 

-  Dehazing  method  efficient  for  human  perception  will  not  be  automatically  the  best  one  for 
automatic  detection  (like  the  best  sensor  in  SWIR  is  not  the  best  for  visible) 

-  Autonomous  technique  cannot  be  done  with  this  technique 

-  There  is  no  discrete  scale  on  the  dehazing  efficiency  (a  dehazing  technique  will  be  good  or  bad) 

On  the  contrary,  quantitative  methods  could  have  all  this  advantages.  The  main  limitation  of 
those  methods  is  that  they  generally  require  information  on  the  dehazed  scene.  This  limitation 
makes  most  of  quantitative  less  useful  for  real-time  identification.  However,  some  quantitative 
techniques  do  not  need  this  dehazed  input.  Some  of  those  techniques  mentioned  in  Appendix  A- 
Performance  metrics  and  on  the  results  sections  can  be  interesting  especially  for  a  fog  algorithm 
assessment  and/or  a  dehazing  quality  measurement. 


3.4  Results  in  a  controlled  environment 

3.4.1  Spectral  selection 

In  this  section  we  will  present  some  of  the  results  obtained  with  the  polarimetric  camera 
operating  in  the  visible  spectrum,  using  the  laboratory  system  (presented  in  3.3.1  Fully 
controlled  labsetup).  They  are  divided  following  the  same  scheme  as  the  dehazing  method 
presented  in  figure  6:  Spectral  selection,  polarization  modulation  and  image  processing. 
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The  diffusive  and  specular  targets  are  observed  under  three  different  illuminations  (red,  green 
and  blue)  and  three  different  concentration  of  scattered  particle  (figure  16). 
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Figure  16:  Evolution  of  the  visibility  depending  on  the  wavelength  and  the  milk  volume  percentage  in  the 

configuration  2  (see  figure  13) 

In  this  figure,  we  can  clearly  note  the  advantage  of  using  a  spectral  selection  during  the 
acquisition.  It  can  be  observed  that  the  best  configuration  to  see  clearly  the  object  is  obtained  for 
a  low  concentration  of  milk  (as  expected)  and  a  higher  wavelength  (red). 

The  size  of  the  scattering  particles  in  the  experiments  ranging  from  0.04  to  0.3pm,  those  results 
are  in  agreement  with  the  result  obtained  using  the  Mie  theory  (see  figure  3). 

3.4.2  Polarization  modulation 

Two  techniques  are  presented  here:  the  linear  polarizer  technique  and  the  extended  degree  of 
polarization  extraction  technique  (see  the  Multispectral  polarization  imaging  solution  section  for 
detailed). 

3.4.2. 1  Using  a  lin  ear  polarizer 

The  light  intensity  image  obtained  through  a  perfect  linear  polarizer  which  axis  lies  at  an  angle  (j) 
with  respect  to  a  horizontal  plane  is  given  by  the  relation: 

(sf +  cos(2rf))5f (  10  ) 
5S‘"“’'Uy,Z)  =  ^ - - - 2 


Wavelength 

Blue  Green  Red 

- >  SWIR 
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The  SALSA  polarization  camera  actually  measures  all  the  Stokes  parameters  (5o,tr3)>  therefore  it 
is  possible  to  simulate  the  effect  a  perfect  linear  polarizer  would  have  had  on  the  image  acquired. 


It  is  assumed  that  only  airlight  contributes  to  the  polarization  signature.  The  light  coming  from 
the  target  is  un-polarized.  The  goal  is  to  minimize  the  contribution  of  the  polarized  light. 
Therefore,  we  can  search  for  the  polarizer’s  orientation  angle  (j)  that  will  give  the  minimum 
intensity  So  poiar  it  will  correspond  to  the  most  efficient  dehazing. 


Figure  17,  shows  the  evolution  of  the  light  intensity  averaged  over  a  Region  Of  Interest  (drawn 
on  the  diffusive  target)  that  would  be  measured  after  a  linear  polarizer  for  various  angle  (j).  Three 
different  position  of  the  light  source  were  considered  (angle  a). 


Figure  1 7:  On  top:  Image  of  the  diffusive  target  obtained  with  biue  iiiumination  and  0.007%  ofmiik  voiume 
concentration.  The  red  rectangie  represents  the  ROI  used  to  measure  the  intensities  vaiues.  Bottom: 
Evoiution  of  intensities  in  the  ROI  that  would  be  obtained  if  a  polarizer  at  an  angle  cf  was  placed  in  front  of 
a  simple  camera  (red  line)  compared  to  the  total  half-intensity  acguired  with  a  simple  camera  (black 

dashed  line). 

We  can  make  the  following  observations  and  comments: 

•  We  do  see  a  modulation  of  the  light  intensity  when  changing  the  polarizer’s  orientation 

•  The  amplitude  of  the  variation  is  up  to  5% 

•  The  light  intensity  minimums  are  obtained  for  different  polarizer’s  orientations  (j)  in  each 
case. 
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Consequently,  this  method  could  attenuate  the  contribution  of  the  haze  particles  by  up  to  5%  and 
requires  an  adaptive  polarization  configuration  that  would  search  for  the  best  axis 
orientation  in  each  case.  Having  an  actual,  mechanical  rotating  polarizer  is  simply  unrealistic 
for  real-time  imaging.  However,  a  Full  Stokes  polarization  imaging  system  combined  with 
adaptive  software  that  automatically  finds  an  optimal  combination  of  linear  polarization  to 
enhance  the  image  would  be  the  best  solution  for  dehazing  using  this  method. 

3.4.2.2  Using  extended  degree  of  polarization  extraction  technique 

To  demonstrate  the  dehazing  efficiency  of  this  method,  an  image  of  the  diffusive  target  was 
acquired  when  no  scattering  medium  was  present  (clear  atmosphere)  and  then  again  when  the 
scattering  particles  medium  was  added  into  the  aquarium. 


Then,  two  of  the  polarization  dehazing  techniques  presented  earlier  were  applied  on  the  image. 
We  tried  to  determine  how  the  contrast  between  the  target’s  edges  and  the  background  evolved. 


Dehazing  with  one  polarizer  Dehazing  with  extended  DOP 


Figure  18:  Top:  Evolution  of  intensities  detected  before  and  after  polarization  dehazing  processing  for  a  0.007% 


milk  volume  concentration  and  green  illumination  in  configuration  2  (see  figure  13).  The  first  image  serves  as  a 


reference  when  there  is  no  scattering  particle  present.  Bottom:  Evolution  of  the  normalized  intensities  values  along 


the  red  dashed  lines  on  the  top  of  the  figure. 


On  the  top  row  of  Figure  18,  we  can  observe  qualitatively  that  the  image  dehazed  with  extended 
DOP  extraction  technique  provide  a  much  better  contrast  than  the  image  with  haze  or  the  image 
dehazed  with  one  polarizer. 
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In  order  to  evaluate  quantitatively  the  improvement  we  measure  the  evolution  of  the  intensity 
along  a  line  on  the  image  crossing  areas  where  the  target  is  and  where  there  is  no  target.  This 
allows  to  define  a  contrast  ratio  between  the  highest  intensity  observed  (measured  on  the  target) 
and  the  lowest  one  (measured  where  there  is  no  target).  We  measure  the  same  contrast  ratio  on 
the  image  without  haze  in  order  to  have  the  reference  of  the  image  that  should  be  obtained  with 
complete  dehazing.  In  this  experiment,  the  presence  of  haze  reduces  the  contrast  ratio  by  a  factor 
8  (figure  18,  between  images  1  and  2). 

Using  the  linear  polarizer  technique  presented  earlier  does  not  increase  drastically  the  contrast 
ratio  (Figure  18,  between  images  2  and  3).  The  extended  DOP  extraction  technique  gives  much 
better  results  (Figure  18,  between  images  3  and  4):  the  contrast  ratio  is  improved  and  no  noise  is 
added  to  the  image.  The  contrast  ratio  between  the  image  dehazed  and  the  regular  image  is 
increased  by  a  factor  2  on  this  experiment. 


3.4.2.3  Image  processing 

On  figure  19,  the  results  of  the  application  of  the  DCP  (dark  channel  prior)  technique  (for  more 
details  please  refer  to  Appendix  B-Post-processing  techniques)  on  the  image  dehazed  with 
extended  DOP  is  provided. 


Reference  image  without  haze  Image  with  haze  Dehazing  with  extended  DOP 


Image  processing 


Figure  19:  Evaluation  of  the  image  processing  dehazing  efficiency  on  the  image  dehazed  with  extended  DOP.  Same 

experimental  conditions  as  figure  18 

Using  this  post-processing  technique  allows  to  increase  the  contrast  ratio  by  a  factor  3  compared 
to  the  image  with  haze  on  this  experiment.  However,  we  can  note  that  even  if  this  image  have 


Topic  Number:  N161-055 


Page  27 


N00014-16-P-2035 


Bossa  Nova  Technologies,  LLC 


FINAL  REPORT 


11  January  2017 


higher  contrast  ratio  than  the  image  dehazed  with  extended  DOP  technique  it  is  noisier  than 
expected  (compared  to  the  reference  image  without  haze). 


Figure  20:  Contrast  ratio  enhancement  on  the  resuits  provides  in  figure  18  and  figure  19.  The  dash  iine  represents 

the  contrast  ratio  measured  without  haze. 

From  this  result  (summarize  in  figure  20),  we  can  conclude  on  the  necessity  of  having  a  fog 
assessment  method  (one  of  which  is  presented  in  the  next  section)  based  on  quantitative 
evaluation  to  determine  when  it  is  judicious  to  use  each  technique. 


3.5  Results  in  a  non-controlled  environment 

In  this  section  we  report  on  the  results  obtained  during  the  outdoor  experiments  described  in  the 
testing  environment  section.  We  used  the  polarimetric  imaging  system  for  the  visible 
wavelengths  as  well  as  a  non-polarimetric  SWIR  camera. 


3.5.1  Ist  Step:  Spectral  selection 

The  same  static  scene  was  observed  through  different  spectral  filter  using  the  polarimetric 
camera.  The  results  are  display  in  figure  21a). 
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Image  400nm 


Image  450 nm 


a 


Image  SSOnm 


Image  600nm 


Image  OSOnm 


Visible  edges  in  the  original  image  450nm:  3980 


Visible  edges  in  the  original  image  SSOnm:  13703 


Visible  edges  in  the  original  image  GOOnm:  22636 


Visible  edges  in  the  original  image  6S0nm:  23132 


Figure  21:  a)  Intensity  of  the  same  scene  under  the  same  ciimatic  conditions  but  observed  through  different  spectrai 
fiiter  and  b)  Image  and  number  of  edges  visible  on  the  ROI  (red  rectangle)  in  the  intensity  image  using  gradient 

ratioing  method 


It  can  be  observed  in  this  experience  (figure  21. a)  that  the  haze  effect  is  less  important  in  higher 
wavelength  (650nm)  than  in  low  wavelength  (400nm). 

In  order  to  evaluate  this  effect  without  scene  knowledge,  we  used  an  edges  detector  tool  on  the 
images  (figure  20.b).  When  haze  is  present,  the  contrast  on  the  image  gets  reduced.  Thus,  if 
the  image  is  segmented  (following  the  definition  of  the  meteorological  visibility  distance 
proposed  by  CIE  {the  International  Commission  on  Illumination}  in  1987)  the  number  of  edges 
detected  gives  a  good  metric  to  evaluate  the  haze  effect.  High  number  of  visible  edge  means 
more  contrast  so,  less  haze  effect.  Quantitatively,  we  can  observe  that  spectral  selection 
increases  by  231  times  the  number  of  edge  visible  (figure  22). 
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Figure  23  represents  the  same  scene  observed  with  the  visible  camera  equipped  with  spectral 
filters  centered  at  400nm  (blue),  650nm  (red)  and  with  the  SWIR  camera.  If  we  focus  on  the 
mountain  edges,  we  can  notice  that  the  image  obtained  with  the  SWIR  in  this  fog  condition  is 
better  that  the  one  obtain  with  the  visible  camera  at  400nm  but  clearly  worse  than  the  one 
obtained  with  the  visible  camera  and  a  filter  centered  at  650nm. 

However,  it  should  be  emphasized  that  the  SWIR  camera  did  not  operate  with  a  specific 
wavelength.  The  images  acquired  are  cumulating  the  contributions  of  the  entire  SWIR,  which 
could  have  destructive  effects  on  the  image. 

For  that  purpose,  we  have  dedicated  Phase  I.B  effort  to  study  the  best  configuration  possible 
within  the  SWIR  and  the  NIR  spectral  bandwidths. 

3.5.2  2"*!  Step:  Polarization  modulation 

On  the  best  image  obtained  with  the  spectral  selection  (visible  camera  at  650nm),  we  applied  the 
polarization  imaging  techniques  described  above  (see  Appendix  B-polarization  modulation  for 
details).  The  result  obtained  is  presented  in  figure  24. 


Polarized  component  of  the  total 
intensity 

Figure  24:  Image  obtained  at  650nm  (figure  20)  dehazed  by  using  the  poiarization  imaging  technigue.  The  image 

dehazed  corresponds  to  the  non-poiarized  intensity. 

We  can  note  that  this  method  further  increases  the  quality  of  the  image.  We  performed  the  same 
evaluation  technique  based  on  the  visible  edges  detected.  The  number  of  edge  recovered  after 
this  polarization  modulation  step  increased  by  113%  compared  to  using  only  the  spectral 
selection. 

3.5.3  3'^d  Step;  Image  processing 

On  the  best  image  obtained  after  the  two  acquisition  steps  (spectral  selection  and  polarization 
modulation),  the  Dark  Channel  Prior  (DCP)  image  processing  tool  was  applied  (Figure  25). 
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Figure  25:  Best  image  obtained  with  spectrai  seiection  and  poiarization  moduiation  technigue  with  and  without 
image  dehazing.  The  iast  row  is  a  zoom  on  the  ROI  area  (red  rectangie) 

By  using  this  technique,  we  have  been  able  to  increase  by  3  times  the  number  of  edges  recovered 
compared  to  using  only  the  acquisition  optimization  steps. 


4  Conclusion 

During  this  phase  I  effort,  successful  demonstration  have  been  done  that  a  dehazing  effect  can  be 
obtained  by  combining  efficiently  (Figure  26): 

-  Spectral  selection 

-  Polarization  modulation 

-  Image  processing 

#  of  edges:  103  23,132  26,063  77,770 


Figure  26:  Decomposition  of  image  dehazing  processing 
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As  it  can  be  seen  above,  the  two  acquisition  steps  (spectral  selection  and  polarization 
modulation)  have  allowed  increasing  significantly  the  quality  of  the  image  acquired.  Then,  the 
post-processing  has  been  efficient  to  complete  the  dehazing  process. 


#  of  edges:  103 


77,770 


14,038 


Only  Image 


processing 


Figure  27:  Comparison  of  the  image  obtained  after  our  method  and  the  one  obtained  using  oniy  image  processing 

This  three  step  technique  has  permitted  to  increase  notably  the  operational  range  of  the  imaging 
system  technology  compared  to  conventional  method  (figure  27).  This  has  been  quantified 
through  the  contrast  of  the  scene:  the  number  of  edges  detected  has  been  increased  755  times 
compared  to  the  same  raw  image  acquired  at  400nm  (Figure  27).  It  is  an  amelioration  by  a 
factor  5.5  compared  to  using  only  post-processing  techniques. 

We  have  successfully  demonstrated  that  combined  results  of  acquisition  improvements  and 
additional  post-processing  have  increase  drastically  the  image  quality  of  outdoor  scene  in 
visible  light. 
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5  Proposed  Additional  SBIR/STTR  Funded  Research 

Three  main  guidelines  have  been  determined  for  the  phase  Ib  and  phase  II  effort  following  the 
good  results  obtained  during  this  phase  I  effort: 

•  On  the  spectral  selection  side,  we  have  validated  the  importance  of  wavelength.  For  the 
fog  used  during  our  experiment  we  have  been  able  to  demonstrate  that  the  best  spectral 
wavelength  to  reduce  haze  effect  is  between  red  (650nm)  and  SWIR  (l-3um).  Our 
validated  theory  also  pointed  out  that  this  best  configuration  will  vary  depending  on  fog 
condition.  We  propose  for  this  reason  to  pursue  and  extend  further  our  studies  in  SWIR, 
visible  and  NIR  during  a  phase  Ib  and  a  phase  II  effort. 

•  On  the  polarization  side,  we  have  been  able  to  validate  various  efficient  methodologies. 
They  present  the  advantage  compared  to  other  existing  techniques  to  be  fast  (once 
implemented,  the  methods  are  instantaneous),  robust  and  easy  to  implement.  We  propose 
for  this  reason  to  pursue  this  evaluation  and  technique  characterization  in  visible  and 
extend  further  our  studies  in  SWIR  and  NIR  during  a  phase  Ib  and  a  phase  II  effort. 

•  On  the  image  processing  side,  many  methods  exist  with  their  advantages  and  their 
inconvenient.  The  method  used  gives  good  dehazing  results.  However,  it  can  in  some 
case  induce  in  noise  and  is  not  suitable  for  real  time  measurement  (around  2minutes  to 
process  one  image  at  a  resolution  of  1024x1024  pixels).  To  overcome  those  limitation, 
the  next  step  planned  for  a  phase  Ib  and  phase  II  effort  is  to  create  a  fog  assessment 
algorithm  (in  order  to  reduce  time  when  fog  is  not  present)  and  combine  it  with  an  image 
processing  technique  (in  order  to  use  faster  algorithm  when  it  will  improve  or  at  least  not 
degrade  the  dehazing  process  efficiency). 
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7  Appendix  A:  Key  Terms 

Key  terms  and  acronyms  used  in  the  report  in  alphabetical  order. 

Airlisht  (or  backscattered  light):  it  is  the  light  coming  from  the  illumination  source,  scattered  by  the 
medium  and  going  directly  to  the  camera. 

Atmospheric  veil  (or  haze  layer):  it  is  expressed  as  R(jir,  y)  =  A{1  —  t{x,y))  where  A  is  the  global 
atmospheric  light,  t  the  transmission  of  the  medium  and  (x,  y)  is  pixels  coordinates  of  the  scene.  It  can  be 
seen  as  a  “veil”  we  are  seeing  through  to  observe  a  scene. 

Desree  of  linear  polarization  (DOLP):  it  is  a  quantity  used  to  describe  the  portion  of  an  electromagnetic 
wave  which  is  linearly  polarized.  A  perfectly  linearly  polarized  wave  has  a  DOLP  of  100%,  whereas  an 
un-polarized  wave  has  a  DOLP  of  0%.  It  can  be  expressed  using  only  the  normalized  stokes  parameters  Si 
and  S2:  DOLP  =  Si  +  S2. 

Image  processing:  it  is  a  post -processing  dehazing  method  proposed  in  this  report.  It  consists  in  reducing 
the  haze  effect  after  the  image  is  acquired.  Common  technique  involves  applying  low-pass  filters  and  area 
recognition. 

Normalized  ( or  reduced)  stokes  parameters:  set  of  the  three  Stokes  parameters  Si,  S2  and  S3  normalized 
by  the  first  Stokes  parameter  So.  They  are  noted  (si,  S2  and  S3)  with  Si=  Si  /So  for  i=l,  2  and  3. 

Performance  metrics:  Any  metrics  that  can  be  used  to  quantify  the  efficiency  of  any  dehazing  method  (in 
term  of  image  recover  and/or  time  processing).  They  can  be  classified  in  two  categories:  qualitative  and 
quantitative.  Qualitative  methods  are  generally  based  on  collection  of  human  perception  on  the  dehazing 
effect  of  a  method  compared  to  another  one.  Quantitative  method  can  be  absolute  (no  image  reference 
needed)  or  relative  (to  an  image  dehazed).  Absolute  quantitative  method  includes  for  instance 
computational  time  evaluation  or  edge  measurement.  Relative  method  includes  for  instance  contrast  ratio 
measurement,  Peak  signal  to  noise  ratio,  mean  square  error. 

Polarization  modulation:  it  is  a  dehazing  method  proposed  in  this  report.  It  consists  in  modulating  the 
polarization  response  of  the  acquisition  system  in  order  to  reduce  haze  effect.  Common  method  used  by 
photographers  consists  in  using  a  simple  polarizer. 

Polarization  of  light:  It  is  a  parameter  applying  to  electromagnetic  waves  that  specifies  the  geometrical 
orientation  of  the  electric  field  oscillation.  It  is  with  the  intensity  and  the  wavelength  a  parameter  used  to 
characterize  completely  light  properties. 

Spectral  selection:  it  is  a  dehazing  method  proposed  in  this  report.  It  consists  in  selecting  specific 
wavelength  in  order  to  reduce  haze  effect. 

Stokes  parameters:  set  of  four  values  (So,  Si,  S2  and  S3  also  noted  I,  Q,  U,  and  V)  that  describe 
completely  the  polarization  state  of  an  electromagnetic  radiation.  Simple  mathematical  operation  allow 
from  those  values  to  determine  parameter  such  as  total  intensity  (I),  (fractional)  degree  of  polarization 
(DOP),  and  the  shape  parameters  of  the  polarization  ellipse. 

Wavelength:  it  is  the  spatial  period  of  the  electromagnetic  wave — the  distance  over  which  the  wave's 
shape  repeats. 
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8  Appendix  B:  Supplemental  Data 
a.  Rayleigh  scattering 


Figure  28:  Scattering  modei  in  Rayieigh  theory 

In  this  regime  a  «  1  which  means  that  the  particle  is  small  enough  compared  to  the  wavelength  so  that  the  local 
electric  field  component  of  light  is  uniform  at  any  instant. 

This  oscillating  electric  field  induces  an  oscillating  dipole  due  to  the  polarizability  ap  of  the  particle: 

p(t)  =  apEo(t)  =  ap(Eo,x„(t)  Xq  +  Eo,y„(t)  yo)  (  U ) 

with  p  dipole  moment,  Eq  incident  electric  field  and  (xO,  yO)  Cartesian  coordinates  of  the  incident  light. 

Using  the  retarded  scalar  V,  and  vector  A,  potential,  the  radiated  electric  field  can  be  expressed: 

Er(r,  t)  =  -  VVr(r,  t)  -  t)  ( 12 ) 

Where  (respectively  is  the  first  (respectively  second)  time  derivative  and  V  is  the  gradient  operator. 

If  we  consider  localized  dipoles  are  in  far-field  (the  observation  point  is  far  away  from  the  haze  particle),  the 
relation  can  be  rewritten: 


1  r 


(13) 


-0 


rc 


+  fio 


rCj.  X  (  rCj.  X  a^2  (  p  j 


\^o 

4nr 

where  Po  the  permeability,  Eq  the  permittivity,  c  the  speed  of  light  and  r  the  distance  between  the  observer  and 
the  particle. 


Using  Eq.ll,  this  equation  can  be  written  more  explicitly  in  the  coordinate  of  the  observer  by: 
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Er(x,y,t) 


ap5f.2  ^Eq  (,)jsin0 


4TTrc^ 


eg 


(14) 


With  0  the  scattering  angle  and  Bq  a  unitary  vector. 

From  this  equation,  we  can  conclude  that  this  dipole  will  radiate  at  the  same  frequency  as  the  applied  field  in  all 
direction  homogeneously. 

Developing  this  equation,  it  is  possible  to  express  the  resulting  intensity  of  light  emitted  by  one  sphere  particle 
with  a  diameter  D  in  far  field  by  the  relation: 


1  +  cos^  9 
2z^ 


lo 


(15) 


with  n  the  refractive  index  of  the  particle,  /q  the  intensity  of  incident  light  on  the  particle,  z  the  distance  to  the 
particle. 

Integrating  this  equation  on  all  angles,  we  can  also  determine  the  scattering  cross  section  efficiency  for  this  model 

UR-Scat  • 

2n^ 

(7R-Scat  =  — 


To  determine  the  degree  of  light  polarization,  we  can  express  the  light  radiated  by  the  particle  in  the  observation 
coordinates: 


Er(x,y,t) 


ap 

4TTrc^ 


X  +  cos  9  df.2 


(17) 


Using  the  Stokes  formalism,  we  can  express  the  polarization  light  through  different  measurable  intensities: 


^detectip^’  y '  '^) 


/; 

Vs; 


0, detect^ 
^1, detect 

r 

2,  detect 

3,  detect^ 


(18) 


with  So  the  total  intensity  of  the  light  detected  by  the  camera.  Si  the  difference  in  intensities  between  the 
horizontal  and  vertical  linearly  polarized  components,  S2  the  difference  in  intensities  between  linearly  polarized 
components  traveling  at  45°  and  -45°  with  respect  to  the  x-axis;  S3  the  difference  in  intensities  between  right  and 
left  circularly  polarized  light. 
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We  can  therefore  express  the  degree  of  linear  polarization  as: 


DOLP  = 


So 


V(||£;,|P-||E^P)  +  (||E+45lP-||g-45lP) 

5o 


(19) 


If  Eq.l7  is  inserted  on  Eq.l9,  we  obtained  the  general  relation  for  the  degree  of  linear  polarization: 


DOLP  = 


sm^9 
1  +  cos^  9 


(20) 


This  relation  on  the  light  polarization  scattered  by  the  particle  is  important  since  it  gives  direct  relation  between 
the  geometry  of  the  illumination-particle-observer  scene  and  the  polarization  of  light  independently  of  the  optical 
characteristics  of  the  scatterer  object  (in  the  limit  defined  above). 

b.  Mie  scattering 

The  Mie  solution  describes  the  scattering  of  an  electromagnetic  plane  wave  by  a  single  homogeneous  sphere  using 
the  Maxwell  equations. 

To  use  this  model,  there  are  some  prerequisites: 

1-  The  quantum  mechanical  effects  are  negligible  and  the  physics  can  be  described  using  classical 
electromagnetism  equations  (Maxwell  equations). 

2-  Haze  particles  will  be  approximated  by  sphere  (this  approximation  can  be  made  due  to  the  high  surface 
tension  of  water). 

3-  The  haze  concentration  is  low  enough  to  consider  that  interaction  between  particles  is  negligible 
(independent  scattering):  there  is  no  interference  between  the  waves  scattered  by  each  particles.  This 
approximation  is  usually  considered  as  obtained  when  the  radius  of  one  particle  is  3  times  less  than  the 
distance  between  one  particle  and  another. 


In  this  theoretical  framework,  we  can  apply  the  Maxwell  equations  for  the  two  medium:  the  haze  particle  and  the 
air.  We  can  this  way  express  the  electric  field  through  the  Helmholtz  equation: 

AE  +  k^n^E  =  0  (21) 

With  n  (respectively  k)  the  index  of  refraction  (resp.  the  angular  wavenumber)  in  the  medium  considered. 


a-  Definition  of  vector  and  scalar  potential 

Since  spherical  object  are  the  subject  of  interest,  we  can  express  this  equation  using  only  the  scalar  propagation  in 
spherical  coordinates  (r,  0,  (p): 


(r2  9  ^  ^  (sm  9 

r^dr\  dr)  r^sm9d9\  50/  r^sm^9d9\  dcp^  j 


+  k^n^cp  =  0 


(22) 
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With: 


E  = 


1  1 
—  Vx  (nkcbr)  H — -  Vx 
nk  nk 


nk 


Vx  (nkcjyr) 


(23) 


The  electric  field  can  also  be  expressed  through  the  vector  potential  A: 


E  = 


d 

dt 


A(r,  t)  —  V0(r,  t) 


(24) 


Using  the  Faraday's  law  of  induction: 


i  I 

Vx  A(r,  t)  =  —  VxE 

Cl) 

Using  Eq.l7  and  19  the  vector  potential  can  be  written  as  a  linear  combination  of  the  electric  field  constitutive 
vectors: 


With  Xa  and  7^  constants. 


A(r,  0  =  Vx  (0r)  +  Ya  Vx  Vx  (0r) 


(26) 


b-  Determination  of  scalar  potential  expression 

We  consider  for  this  demonstration  the  case  of  a  sphere  with  a  diameter  D,  a  complex  refractive  index  m  =  n  +  ix 
and  a  conductivity  ji  placed  in  a  nonconductive  environment  (air)  with  a  refractive  index  n2=l. 

An  electromagnetic  plane  wave  with  wavelength  is  incident  on  the  sphere. 


Incident  plane  wave  (Ei„|^ 


Figure  29:  Scattering  modei  in  Mie  theory 


In  this  model,  the  incident  and  scattered  waves  are  decomposed  in  spherical  vectorial  functions. 

For  simple  geometry,  the  separation  of  variables  technique  can  be  used  to  solve  this  equation:  the  solution  can  be 
decomposed  in  three  different  functions  with  dependence  in  only  r,  6  or  rp. 

(p{r,9,(p)  =  f{r)g{9)h{(p)  (27) 
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Using  Eq.22  and  23,  the  solution  can  be  expressed  in  series  of  Ricatti-Bessel  function  (where  J|  and  Y|  are  the  first 
order  Bessel  functions  of  respectively  the  first  and  second  kind)  and  Legendre  polynomials  (where  is  the 
Legendre  polynomial  of  I  order  and  m  kind)  as: 

00  I 


1=0  m=0 
I 


2 


p[^\cos9)  sin(m^) 


1=0  m=0 


(cos  9)  cos(m^) 


(28) 


With  a*”^*,  X|,  yi  constants  and  0^,  0g  odd  and  even  scalar  solutions. 

In  those  equations,  we  can  note  that  Yi+^j  is  divergent  in  the  limit  r  tend  to  0  which  is  a  non-physical  solution  so 
two  case  can  be  defined: 

Inside  the  sphere:  yi  =  0  (spherical  Bessel  functions). 

Outside  the  sphere:  we  can  select  different  parameters  for  X|  and  yi.  Here  we  will  use  xpl  and  yp-i  (Hankel 
functions).  It  corresponds  to  an  outgoing  spherical  wave  pattern  of  the  scattered  field. 


(1  ^ 

To  simplify  the  notation,  we  introduce  the  parameter  z,  (fcr)  = 
set  of  equation  can  now  be  rewritten  as: 


+yi  F,^i(/cr) 


1+^ 


.  The  previous 


“  '  (29) 

00  =  2^  z2\kr)  p[^\cos  9)  sm(m(p) 

1=0  m=0 
00  I 

0g  =  ^  ^  z2\kr)  p[^\cos  9)  cos(rn(p) 

1=0  m=0 


c-  Determination  of  electric  field  expression 
Maxwell  equations  states  that: 


E  = 


Vx(//)  = 


1 

—in'^k 


1 

Vx( - Vxi4(r,  t)) 

i(jL)fi 


(30) 


Using  this  equation  and  Eq.20,  the  incident  fields  can  be  written  as: 


E  = 

*-•  1-nr 


k 


00  I 


y  y  Zi.^Vx  (0r)  +  (0^) 


Z_i 


k 

0)  fi 


1=0  m=0 
00  I 


Hinc  =  — Z  Z  J  ^ 

(=0  m=0 


(31) 


With  Xijfi  and  V)  characteristic  coefficient  of  the  incident  beam.  For  an  incident  plane  wave,  those  coefficients 
can  be  calculated  by  integration  around  of  the  electric  field  around  small  area  and  is  equal  to: 
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Xi,m  = 


21  +  1 

°/(Z  +  1)  ' 

0,  m  +  1 


m  =  1 


(32) 


We  can  express  with  the  same  technique  the  scattered  and  internal  fields: 

00  / 

k  ,  2Z  +  1 


^scat  ,  2 


k 


1  =  0 


a,Vx(0r)  +  th,Vx^-Vx(0r) 


K  21  + 1  f  ( 1  \  '' 


(33) 


f/r,/-  = 


‘int  ^  7 

co^efj. 


1  =  0 
00 

^  V  2Z  +  1 

/  (  ^0  /r;  I  1  ^ 

cu  nZ—i  t(t  +  1) 

(=0 


2Z  +  1  /  /I 

QVx(^,r)  +  id,Vx(-Vx»r) 


2Z  +  1  /  /I 

C;Vx  —  Vx  (0r)  1  +  i  d;Vx  (0r) 


nk 


with  ai,  b|  the  scattering  coefficients  and  C|  and  d|  the  internal  field  coefficient. 


More  specifically,  we  can  completely  express  the  scattering  electric  field  in  the  scattering  plane  using  Eq.  29  and 
31: 


With 


,ikr 


Escatir,9,(p)=— {-COS  (pS2icos  6)  \Eo 


sin  ^  Si  (cos  9) 
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with  7r;(cos  0)  = 


T(i  +  1) 
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dpf'^^Ccose)  ^  r  dpf'^^Ccose) 

— - - and  XiicosO)  =  cosO— - 


d  (cos  0) 


d  (cos  0) 


sin^  0 


2  d^p}-‘"^(cos  9) 


d  (cos  0)2 


(34) 


d-  Mie  scattering  parameters 

To  completely  solve  the  scattering  problem,  it  is  necessary  to  find  the  four  constants  of  the  wave  function 
equations:  ai,  b|,  C|  and  d|. 

To  do  so,  using  the  Faraday's  law  to  a  small  rectangular  loop  whose  long  sides  run  parallel  to  the  interface  between 
two  materials  we  can  determine  that  the  tangential  component  of  the  electromagnetic  field  is  continuous  at  an 
interface.  Thus,  the  tangential  component  (cf.  Figure  26)  of  the  field  inside  the  particle  at  the  limit  (r=D/2)  is  equal 
to  the  sum  of  the  tangential  components  of  the  scattered  and  incident  light  field  (E  and  H): 

Escat,0  T  ^inc.e  ~  (  35  ) 

^scat.O  T  Elincfi  —  ^int,9 

EsCat,<p  T  Eiyic,(p  —  Eiyi^  tp 
^ scat,ip  T  ^inc,<p  —  t^int,<p 
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We  can  then  deduce  the  Mie  coefficient  for  scattering  and  internal  field: 

^  ji(mk£)/2)[(k£)/2))  ii(kD/2)]'  -  /iJi(kP/2)[(mkZ)/2)ji(mk£)/2)]'  (36) 

'  m2  j,(mk£)/2)[(k£)/2))  fip^(k£)/2)]'  -  Aiifi[^^(k£)/2)[(mk£)/2)j,(mk£)/2)]' 

^  _  Ml  ji(mk£)/2)[(k£)/2))  ji(k£)/2)]'  -  ji(k£)/2)[(mk£)/2)ji(mk£)/2)]' 

'  Ml  ji(mk£)/2)[(k£)/2))  fip^(k£)/2)]'  -  fip^(k£)/2)[(mk£)/2)j,(mk£)/2)]' 

^  Ml  jTkP/2)[(kg/2))  fi[^^(kP/2)]'  -  Ml  fi[^^(kg/2)[(kP/2)jTkP/2)]' 

'  Ml  j;(mk£)/2)[(k£)/2))  fip^(k£)/2)]'  -  fip^(k£)/2)[(mk£)/2)j,(mk£)/2)]' 

^  _  Mimh(kP/2)[(kg/2))fip^(kP/2)]'-Mimfi[^^(kP/2)[(kg/2)M(kP/2)]' 

'  m2  j,(mk£)/2)[(k£)/2))  fip^(k£)/2)]'  -  Milip^(k£)/2)[(mkZ)/2)j,(mk£)/2)]' 

With  m  complex  refractive  index  of  the  particle  and  Mi  the  ratio  of  the  magnetic  permeability  of  the  sphere  to  the 
magnetic  permeability  of  the  ambient  medium  (air) 

e-  Scattering  efficiency  and  light  polarization 

Using  Eq.34  and  36,  it  is  possible  to  completely  describe  the  scattering  electric  field  and  determine  parameters 
such  as: 

•  Scattering  efficiency  determined  by  integrating  the  scattered  power  over  all  directions: 

00 

Qscat  = 

1=0 

•  Scattered  light  polarization:  Since  we  determine  completely  the  light  field,  we  can  recover  any  polarization 
light  information  on  the  scattered  light  using  this  model. 

•  Other  parameter  could  be  of  recover  such  as  the  absorption  efficiency  which  are  not  presented  in  this  report; 

In  fog,  a  photon  can  be  scattered  multiple  time  before  reaching  the  detector.  To  describe  this  multiple  scattering 
event,  we  need  to  introduce  a  new  parameter  the  reduced  scattering  coefficient.  It  corresponds  to  the  number  of 
scatter  event  N  observed  during  the  propagation  through  a  homogeneous  scattering  medium  with  a  thickness  d: 

Ps'  =  N/d  and  can  be  expressed: 

[Is  =  Ps  (1  -  g) 

With: 

•  g=<cos(6)>  the  anisotropy  of  scattering  defined  with  a  probability  of  scattering  in  the  forward  direction  and  6 
the  angle  of  the  scattered  light  with  respect  to  the  incident  one:  g=0  corresponds  to  isotropic  scattering  while 
g  ->  1  corresponds  to  high  probability  of  forward  scattering  and  thus  less  diffusion  in  the  medium  composed 
of  many  spherical  particles. 

•  Ps  correspond  to  the  scattering  coefficient  of  one  particle  and  is  defined  as: 

Ms  “  Q-sca  ^  ^particle  ^  ^particle 

Where: 

Qsca  is  the  scattering  efficiency 

Oparticie=Tir^  is  the  geometrical  cross  section  area  of  the  particle  (with  r  the  radius  of  the  particle) 

Cparticie  is  the  Concentration  of  particle  in  the  medium. 

c.  Polarization  modulation 
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One  of  the  causes  of  degradation  in  scattering  medium  is  associated  to  airlight.  Since  the  airlight  is  due  to  light 
scattered  by  small  particles  inside  the  medium,  it  depends  on  the  distance  between  the  object  and  the  camera.  To 
perform  dehazing  using  polarization  modulation,  two  assumptions  have  been  made: 

Scattering  will  induce  mainly  additive  airlight  and  attenuation  of  the  signal  of  interest:  we  neglected  any 
image  blur  or  other  effect. 

Light  directly  reflected  by  the  object  is  mainly  not  polarized 

The  last  approximation  is  valid  for  diffusive  surface  and  can  be  assumed  for  specular  surface  far  enough  (figure 
30). 

The  principle  for  dehazing  is  that  images  recovered  are  the  addition  of  two  images  with  different  weight:  an  image 
of  the  scene  without  haze  and  an  image  composed  of  only  airlight.  The  idea  is  to  reduce  the  last  component  and 
maximize  the  first  one.  To  do  so,  we  present  hereafter  three  different  techniques  based  on  polarization. 

One  virtual  polarizer 

Using  the  Stokes  parameters  measured  with  Bossa  Nova  Technologies'  polarimetric  camera,  it  is  possible  to 
contrast  the  effect  induced  by  a  perfect  linear  polarizer  with  that  of  the  intensity  observed  with  a  standard  camera 
(So,  Polar)-  The  light  intensity  image  after  a  perfect  linear  polarizer  which  axis  lies  at  cp  angle  with  respect  to  a 
horizontal  plane  will  be  given  by  the  relation: 

{So,detectix,y,^)  +  COs(2</))  Si  detectix.y,^)  +  sin(2</))  S24etectix,y,  ) 

^0,Polari_^>  y >  ^ 

Since  we  assumed  the  polarization  light  is  only  due  to  airlight,  we  can  estimate  the  best  configuration  for  the 
polarizer  to  have  the  highest  dehazing  effect  by  calculating  the  min  So,  polar  lor  any  </>  value. 


Dehazing  using  Degree  Of  Polarization 

The  degree  of  polarization  measured  for  a  diffusive  target  is  only  due  to  the  scattering  medium.  Thus,  measuring 
only  the  non-polarized  light  component  will  remove  partially  the  haze  on  the  images  acquired.  This  can  be  done 
using  the  Salsa  camera  through  the  relation: 


-polar(.^’y')  y) 


Si  +  Si  +  Si 


(38) 


The  extended  Degree  of  Polarization  technique 

In  this  dehazing  technique,  the  idea  is  to  remove  the  haze  by  first  mapping  the  distance  of  the  objects.  The  reason 
is  that  airlight  intensity.  A,  depends  on  the  fog  distance  between  the  viewer  and  the  object  through  the  relation: 

A{z)  =  A„[l  -  e- '*  W''"]  *  ’’ ' 

With  (3  the  coefficient  of  extinction  due  to  scattering  and  absorption,  t(z)  =  e~-^o  transmittance  of 

incoherent  light  and  Aoo  the  airlight  of  an  object  which  would  be  at  an  infinite  distance. 
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The  fog  can  be  considered  as  homogeneous  regarding  the  distance  which  means  that  we  can  rewrite  the  previous 
equation: 


i4(z)  =Aoo[l-e  =  i4oo[l  -  f(z)] 

The  total  intensity  detected  I^etect  which  will  be  seen  by  the  viewer  can  be  expressed: 

^detect  ect  +  A{z) 


In  the  assumption  considered  here,  only  airlight  is  partially  polarized.  Thus,  the  airlight  intensity  component  can  be 
expressed  for  each  pixel: 


A(^x,y^  DOP fiiyiigfii  AI{x,y^ 


With  Al  the  difference  of  light  detected  after  a  polarizer  in  two  orthogonal  configurations  where  the  intensity  of 
light  detected  are  extremal  and  DOPairnght  the  degree  of  polarization  of  airlight. 


Red  illumination  (a  =  80°) 


Green  illumination  (a  =  80°) 


Blue  illumination  {n  =  80°) 


Figure  30:  Evolution  of  intensities  that  would  be  obtained  if  a  polarizer  at  an  angle  4>  was  placed  in  front  of  a 
camera  (plotted  lines  on  the  charts)  for  different  region  of  interest  and  different  illuminations.  The  3  rectangles 
represent  the  ROIs  used  to  measure  the  intensities  values.  The  volume  concentration  of  milk  in  this  experiment  is 

0.007%. 
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We  can  note  in  the  figure  above  that  the  intensity  detected  and  the  value  of  Al  are  highly  dependent  of  the  part  of 
the  scene  observed  which  can  easily  be  understand  since  they  depend  on  the  object  radiance  observed  and/or  the 
transmittance  of  incoherent  light  through  fog.  However,  we  can  note  that  the  best  and  worst  polarizations  are 
found  for  the  same  polarizer  angle.  This  is  consistent  with  the  approximation  made  at  the  beginning  that  light 
polarization  is  coming  only  from  the  light  scattered  by  particle:  light  reflected  by  a  specular  object  has  no  reason  to 
be  polarized  at  the  same  angle  as  light  scattered  by  particle. 

To  dehaze  the  images,  it  is  necessary  to  determine  Pz  experimentally  and  so,  Aooand  DOPairiight-  Two  heuristic 
methods  can  be  used  to  determine  those  parameters,  depending  on  the  image  content: 

There  is  a  known  point  where  only  airlight  is  present  (clear  sky  for  example).  In  this  case,  A^oCorresponds 
directly  to  the  measured  intensity  on  this  point  and  DOPgiriight  can  directly  be  determined  using  Eq.42. 

There  is  the  same  object  (for  example  same  tree)  in  at  least  two  different  distance  Zj  and  Zj.  This  method 
can  be  generalized  to  known  object  taken  on  clear  and  hazing  day. 

In  this  report  we  use  the  first  heuristic  method  to  determine  Aoo  and  DOPairiight-  Then,  the  intensity  of  the  object 
without  haze  can  be  expressed  in  each  pixel  as: 


^ object(.^>  y) 


^ detect(.^>  ^ airlight 

1-A(x,y)/A^ 


(43) 


In  this  equation  we  can  note  that  if  there  is  no  fog  present,  lobject(x<y)  =  ldetect(x<y)  since  Al(x,  y)  will  be 
equal  to  zero  and  A(x,y)  =  A^^. 

d.  Post-processing  techniques 

Many  post-processing  techniques  are  available.  They  generally  differ  on  the  filters  used  (bilateral  filter, 
wiener  filter  . . .),  the  size  of  the  filters,  its  adaptability  and  the  speed  of  the  image  processing. 

This  section  will  be  dedicated  to  the  method  used  in  this  report:  the  Dark  Chanel  Prior  Technique  and 
more  specifically  when  it  is  applied  to  a  monochrome  image. 

As  described  in  the  Multispectral  polarization  imaging  solution-  post  processing  techniques  section,  the 
goal  in  this  image  post  processing  technique  is  to  solve  the  under-constrained  following  equation: 

ldetect(x,y)  =y(x,y)t(x,y) -P  P(x,y)  (44) 

WithVix,y)  =  A{l-t{x,y}} 

In  order  to  find  J,  the  idea  is  to  determine  first  the  atmospheric  V. 

The  variation  of  the  values  of  V  is  by  definition  dependent  solely  on  the  depth  of  the  objects.  Thus, 
objects  with  the  same  depth  will  have  the  same  value  of  V  and,  in  many  situations  it  changes  smoothly 
across  small  local  areas.  Thus,  we  can  make  the  assumption  that  atmospheric  veil  has  no  specific  edges  or 
textures.  So,  the  atmospheric  veil  from  the  hazy  image  can  be  recovered  by  applying  a  low  pass  filter  to 
the  image  (texture  being  seen  as  noise).  For  this  technique,  the  filter  used  is  soft  matting  based  on  matting 
laplacien  matrix.  A  bilateral  filter  is  then  applied  to  smooth  small  seal!  textures  images. 
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Two  heuristic  methods  can  be  used  then  to  determine  the  airtight  parameters  for  the  monochrome  image, 
depending  on  the  image  content: 

-  There  is  a  known  point  where  only  airtight  is  present  (clear  sky  for  example).  In  this  case, 

Aoo  corresponds  directly  to  the  measured  intensity  on  this  point  and  DOPaMight  can  directly  be  determined 
using  Eq.42. 

-  There  is  the  same  object  (for  example  same  tree)  in  at  least  two  different  distance  Zi  and  Z2.  This  method 
can  be  generalized  to  known  object  taken  on  clear  and  hazing  day. 

In  the  experiments  used,  we  used  the  first  method:  laetecti^sky^ysky)  =  d 
The  image  dehazed  can  be  then  expressed  as: 


J(x,y)  = 


^detect  (x,y)  -4(1  -  t(x,y)) 


max(t(x,y),  tg) 

With  tO  an  arbitrary  correction  value  (put  to  0.1  in  our  case)  in  order  to  increase  the  visual  intensity  of  the 
image  observed. 
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